The present invention relates generally to data transmission, and specifically to transmission of audio data over multiple-node networks.
The Internet is widely used for transmission of data in various forms. In most cases, the Internet and other multi-node communication networks provide best-effort service, in which the transmitted data is divided into a plurality of packets which are sent over the network without any guarantee as to the transmission time or successful delivery of the transmission to its destination. Thus, some of the packets may not arrive, and others may arrive out of the order in which the packets were sent.
Many Internet sites provide audio data which may be downloaded and played back by a remote user""s computer. The user""s computer can wait until all the data is received before playing the audio data, and if some of the packets are not received, the user""s computer can request that they be re-transmitted. However, when the audio data is provided on-line, for example, when the audio data includes radio transmissions or telephone conversations, the data cannot be re-transmitted.
U.S. Pat. No. 4,538,259, which is incorporated herein by reference, suggests using a buffer at the receiving computer in which audio data is accumulated before it is played. The buffer allows waiting a fixed amount of time for latecoming packets, thus allowing a higher percentage of the sent packets to arrive at the receiving computer in time to be played. However, as stated in the ""259 patent, use of such a buffer causes a delay between receiving the audio data and its playback, a delay which may be very annoying, especially in telephone conversations. It is noted that if the size of the buffer is minimized, the delay is also minimal, but the effectiveness of the buffer is also reduced.
Therefore, the ""259 patent suggests delaying a first packet of a conversation at the receiving computer and handling the rest of the packets of the conversation at set intervals from then on. Thus, the rest of the packets are handled as if they arrived early, and extra time is allotted for unexpected delays. The first packet is sent at a high priority so that the unexpected delay of the packet is minimized. However, this method does not handle cases in which the delay causes a first transmitted packet to be received after a succeeding packet.
U.S. Pat. No. 4,607,363, which is incorporated herein by reference, describes a buffer device for transmission of voice signals over a network. The buffer regulates the traffic which is to be transmitted. When the buffer is filled, a number of packets from the buffer are purged. The size of the buffer and the number of purged packets are adjusted with time.
It is an object of some aspects of the present invention to provide improved methods and apparatus for transmission of real-time audio signals over multi-node communication networks.
It is another object of some aspects of the present invention to provide methods and apparatus for prevention of loss of real-time audio packets transmitted over multi-node communication networks.
It is yet another object of some aspects of the present invention to provide methods and apparatus for receiving and playing real-time audio packets transmitted over a multi-node communication network.
In preferred embodiments of the present invention, a real-time audio receiving unit connected to a network includes an adaptive buffer in which received audio data packets are stored before they are played. The packets are delayed in the buffer for a time dependent on the size of the buffer. The size of the adaptive buffer is adjusted according to characteristics of the network, preferably including a jitter level of transmission of packets through the network. Preferably, when the network has a high jitter level, the delay is set to be relatively long, whereas when the network has a low jitter level, the delay is preferably set to be relatively short. Preferably, the size of the buffer is readjusted each time a data packet is read out from the buffer and/or a data packet is inserted to the buffer.
In some preferred embodiments of the present invention, when the characteristics of the network require enlarging of the buffer, packets of audio data representing silence are inserted into the buffer. Preferably, the silent packets are evenly inserted between the words of the received audio data in the buffer, such that the insertion will be substantially unnoticeable to a listener. Preferably, when the characteristics of the network allow reducing the size of the buffer, the size of the buffer is reduced by removing silent fragments of data and/or recesses which have been left for packets which have not arrived. The discarded fragments are preferably removed from the portion of the buffer closest to being played. Alternatively or additionally, some of the silent fragments are removed from between the words in the buffer in such a manner as to leave the buffer with even spacing of silence between the words. Alternatively or additionally, the size of the buffer is reduced, when necessary, by discarding packets of data.
Alternatively or additionally, the size of the buffer is dynamically adjusted by changing the time used to play each received audio data packet. Preferably, the time of playing the audio data packets is adjusted without affecting the pitch of the audio data, preferably using a Time Domain Harmonic Scaling (TDHS) method, as is known in the art.
There is therefore provided in accordance with a preferred embodiment of the present invention, a method of providing audio packets received from a network to a speaker which plays the packets, including accumulating the received audio packets in a buffer having a variable size, calculating a parameter indicative of a feature of data flow in the network, adjusting the size of the buffer responsive to the parameter, and providing the speaker with audio packets from the buffer.
Preferably, accumulating the packets includes storing the packets in a buffer defined by two pointers, and adjusting the size of the buffer includes moving at least one of the pointers.
Preferably, accumulating the packets includes storing the packets in the buffer responsive to ordering labels in the packets.
Preferably, providing the speaker with the audio packets includes providing the packets substantially independently of a rate of accumulating the packets in the buffer.
Preferably, providing the speaker with the audio packets includes providing the packets after a minimal delay which is dependent on the size of the buffer.
Preferably, calculating the parameter includes calculating the parameter responsive to the accumulation of the received audio packets.
Alternatively, calculating the parameter includes calculating the parameter substantially irrespective of the accumulation of the received audio packets.
Preferably, calculating the parameter includes calculating a jitter of the network.
Preferably, calculating the jitter includes calculating a change in the size of the buffer in two consecutive measurements of the buffer size.
Preferably, calculating the parameter includes calculating the parameter substantially each time the size of the buffer changes due to insertion of a packet thereinto or removal of a packet therefrom.
Preferably, adjusting the size of the buffer responsive to the parameter includes determining a watermark representing a desired size of the buffer.
Preferably, determining the watermark includes changing the watermark if the parameter is not within a desired range of the watermark.
Preferably, determining the watermark includes enlarging the watermark when the data flow in the network is relatively irregular and reducing the watermark when the data flow is relatively regular.
Preferably, adjusting the size of the buffer includes adding or removing one or more leading silent packets in the buffer.
Preferably, adjusting the size of the buffer includes adding or removing one or more silent packets between audio packets representing words in the buffer.
Preferably, adjusting the size of the buffer includes changing the time required to play at least some of the audio data in the buffer.
Preferably, changing the time required to play the audio data includes changing a speed of playing the audio data substantially without effecting the pitch of the resultant audio signals.
There is further provided in accordance with a preferred embodiment of the present invention, apparatus for providing audio packets received from a network to a speaker which plays the packets, including a processor which accumulates the received audio packets in a buffer having a variable size, calculates a parameter indicative of a feature of data flow in the network, adjusts the size of the buffer responsive to the parameter, and provides the audio data to the speaker.
Preferably, the processor defines the buffer using two pointers and adjusts the size of the buffer by moving at least one of the pointers.
Preferably, the processor provides the audio packets to the speaker substantially independently of a rate of accumulating the packets in the buffer.
Preferably, the parameter includes a jitter.
Preferably, the processor updates the parameter each time the size of the buffer changes.
Preferably, the processor determines a watermark which represents a desired size of the buffer.
Preferably, the processor changes the watermark such that the parameter will substantially always be within a desired range of the watermark.
Preferably, the processor enlarges the watermark when the data flow in the network is relatively irregular and reduces the watermark when the data flow is relatively regular.
Preferably, the processor adjusts the size of the buffer by adding or removing one or more leading silent packets in the buffer.
Preferably, the processor adjusts the size of the buffer by adding or removing one or more silent packets between audio packets representing words in the buffer.
Preferably, the processor adjusts the size of the buffer by changing the time required to play at least some of the audio data in the buffer.
There is also provided, in accordance with a preferred embodiment of the present invention, a computer program product having computer readable program code embodied therein, which code causes a computer receiving audio packets over a network to accumulate the received audio packets in a buffer having a variable size and to calculate a parameter indicative of a feature of data flow in the network and adjust the size of the buffer responsive to the parameter, wherein the computer conveys the audio packets to a speaker which plays the packets.
The present invention will be more fully understood from the following detailed description of the preferred embodiments thereof, taken together with the drawings, in which: